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Abstract 

A backward biorthogonalization approach is proposed, which modifies biorthog- 
onal functions so as to generate orthogonal projections onto a reduced subspace. 

The technique is relevant to problems amenable to be represented by a general 
linear model. In particular, problems of data compression, noise reduction and 
sparse representations may be tackled by the proposed approach. 

I. INTRODUCTION 

We introduce a backward biorthogonalization technique relevant to the general linear model. 
Any data which may be described in terms of a linear combination of waveforms satishes this 
model. For instance, the response of a physical system to a particular interaction varying as 
a function of a parameter, say t, is often represented by a measurable quantity /(t), which is 
amenable to be expressed in the fashion: 

N 

fit) =J2^n(^n{t), (1) 

n 

where the model waveforms an] n = 1,... ,N are derivable by recourse to physical consid¬ 
erations. The determination of the coefficients Cn ] n = 1,..., N entails to solve the inverse 
problem when the function f{t) is measured. The superscript N indicates that, unless the 
waveforms an are orthogonal, the appropriate coefficients df depend on the order of the model, 
i.e., the number N of waveforms being considered in ([^). If such waveforms are linearly inde¬ 
pendent, then, there exits a set of reciprocal functions ; n = 1,... ,N which is biorthogonal 


1 



to the former, i.e. {a^ \am) = ^n,m IHJI]- Here the superscript N indicates that the biorthogonal 
functions allow for constructing orthogonal projections onto the subspace Vn spanned by N 
waveforms Hence, the coefficients of the linear expansion (|^ approximating a function 
/ at best (in a minimum distance sense) can be obtained by computing the inner products 
= {a^\f) @,§. 

Since the reciprocal set ; n = 1,... ,iV (and therefore the coefficients c^) depend on the 
number N of total waveforms, they should be recalculated when this number is enlarged or 
reduced. This feature of non-orthogonal expansions is discussed in p|,H,^, where a recursive 
methodology is introduced for transforming the reciprocal set ; n = 1,..., into a new 
one ; n = 1,..., iV + 1. The latter is guaranteed to yield orthogonal projections onto the 
subspace Idv+i arising by the inclusion of a waveform a^+i in Vn, i.e., Vn+i = Vv + aN+i- 
Here we wish to consider the converse situation: Let us suppose that the reciprocal waveforms 
; n = 1,..., iV are known and we want to modify them so as to obtain orthogonal pro¬ 
jections onto a subspace Vn/oj arising by eliminating an element, say the j-th one, from Vn- 
Then, Vn/oj = spanja^ ; n = 1,..., j — 1, j -|- 1,..., A^}. Our aim is to construct the corre¬ 
sponding reciprocal functions ; n = 1,..., j — 1, j + 1,..., N hj modifying the previous 
] n = 1,... ,N. 

Let us suppose that in the summation of (|^) we want to retain only some terms and approxi¬ 
mate f{t) by a linear combination of those elements. Thus, to obtain the best approximation 
of f{t) by a linear combination of such a nature, we need to recalculate the coefficients corre¬ 
sponding to the waveforms we wish to retain. If we simply disregarded the coefficients of the 
unwanted terms, but did not recalculate the remaining ones, the approximation would not be 
optimal in a minimum distance sense. The approach we propose in this Communication allows 
for the necessary modifications of coefficients so as to achieve the optimal approximation. The 
method is based on an iterative technique capable of adapting biorthogonal functions in order 
to generate orthogonal projections onto a reduced subspace. 

The paper is organised as follow: Sections H introduces the notation, discusses the motivation 
to the proposed approach and summarises a previously introduced forward biorthogonaliza- 
tion method Section HI discusses the proposed biorthoganization technique to transform 
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biorthogonal functions in order to build orthogonal projections onto a reduced subspace. The 
conclusions are drawn in Section IV. 


II. NOTATION, BACKGROUND AND MOTIVATION TO THE APPROACH 


Adopting Dirac’s vector notation we represent an element / of a Hilbert space as 
a vector |/) and its dual as (/|. Given a set of ^-normalized continuous orthogonal vectors 
{\t) ; —oo < t < oo ] {t\t') = 6(t — t')}, the unity operator in H is expressed 

4= lim f \t){t\dt. (2) 

T^oo J-T 

Thus, for all |/) and \g) G H, by inserting Ih in {f\g), i.e, 

(/|4|£/) = lini / {f\t){t\g)dt (3) 

T^oo J-T 

one is led to a representation of in terms of the space of square integrable functions, with 
{Ad) — di't) and {g\t) = (Ad)* = d*{A) where g*{t) indicates the complex conjugate of g{t). 

Let vectors |a„) E H ] n = 1,..., cx) be a Riesz basis for H. Hence, all |/) G H can be 
expressed as the linear span 

OO 

\f) = J2^n\ar,) (4) 

n=l 

and there exists a reciprocal basis |d„) ; n = 1,..., oo for 7^ to which the former basis is 
biorthogonal i.e., {Q.n\otm) = 5n,m 111 - The reciprocal basis allows to compute the coefficients Cn 
in @ as the inner products 

Cn = (dn|/) = lim f a*^{t)f{t)dt. (5) 

T—>oo J-T 


Thus, 

OO 

I/) = H l«n)(ttn|/) (6) 

n=l 

so that, by denoting 

OO 

I ^ ^ ( 7 ) 

n=l 
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(Q) can be recast as f = If, which implies that J is a representation of the identity operator 
in H and we have the following generalization of the Plancherel-Parseval identity 

OO 

iii/>ir={/i/i/> = E5:<^» (8) 

71=1 

with Cn as in (I) and c; = {f\an). 

If the basis |a„); n = 1,..., cxd is orthogonalized and we denote by |'0n); n = 1,..., cx) the cor¬ 
responding orthogonal vectors after normalization to unity, then the new basis is self-reciprocal, 
i.e., it satisfies the orthonormality condition {'fml'f’n) = ^m,n and provides a representation for 
the identity operator as given by 

OO 

(9) 

n=l 

This representation of the identity operator can be seen as a particular case of (|^, by considering 
the basis and its reciprocal identical to |'0„); n = 1,..., cx). The equivalence between 0 and 
d^) holds only when both sums run to infinity. Because, on the one hand if the sum in (P) is 
truncated up to N terms we obtain an operator, P, given by 

N 

P=Y.\^n){^nl (10) 

n=l 

which is the orthogonal projector onto the subspace Vat spanned by N vectors |a„); n = 
1,..., iV. On the other hand, by truncating (0) up to N terms one obtains an operator 

N 

Q ^ ) |Q^n)(c>^n|) (11) 

n=l 

which is idempotent, and hence a projector, but as it fails to be self-adjoint it is not an 
orthogonal projector operator. As a consequence, the approximation of |/) that we obtain by 
truncating the expansion (|I|) up to N terms is not the best approximation of |/) that can be 
obtained by a linear superposition of N vectors |a„). If one wishes for orthogonal projections by 
means of biorthogonal families, then biorthogonal vectors ] n = 1,... ,N specially devised 
for such a purpose must be constructed. The superscript N indicates that if the subspace Idv 
is enlarged (or reduced) each function should be recalculated. 
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A. Forward Adaptive Biorthogonalization 


Let \an) be a set of linearly independent vectors and let vectors be obtained by orthog- 
onalizing the formers in such a way that |'0n) = \otn) — Ay-i l^n), where iV„_i is the orthogonal 
projector operator onto the subspace 14-i spanned by \ai ); Z = 1,..., n — 1. Then, it is proved 
in that vectors \ot^^) arising from l-^i) = |q!i) through the recursive equation: 


a 


k-\-l 


) = \^n) 


l^fc+l) 


-(afc+i|d^) 


n = 1,..., Zc 


|aSl> = 


li'fc+i> 


( 12 ) 


are biorthogonal to vectors |q(„) ; n = 1,..., k+1 and provide a representation of the orthogonal 


projection operator onto 14+i i.e., 


k+1 


fc+1 


a-w. = E l“»){ar‘l = pL, = E |av‘)KI- 


(13) 


n=l 


n=l 


As discussed in |^, in order to reduce numerical errors the vectors \'ipk) are conveniently com¬ 
puted by Modified Gram Schmidt procedure or Modified Gram Schmidt with pivoting |Q,^. 
Since the unique vector in 14+i minimizing the distance to an arbitrary vector |/) G 7Y is ob¬ 
tained by the operation Pvk+ it follows from (P!3|) that the coefficients of the linear 

expansion 


/c+1 

(14) 

n=l 

which approximates an arbitrary |/) G at best in a minimum distance sense, can be recur¬ 
sively obtained as: 

T' = fS-(a;;|a»+.)||^^ ; n = l,...,k 

t+l ^ (fe+ll/) 

IIWt+i>IP’ 

Cl = fSlT- 

This technique, yielding forward approximations, has been shown to be of assistance in sparse 
signal representation by waveforms selection as well as data set selection [Q. Nevertheless, 
in those and other application areas, it is clear the need for a technique yielding approximations 
in the opposite direction. Hence the motivation to the approach of the next section. 
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III. BACKWARD ADAPTIVE BIORTHOGONALIZATION 


Let denote the subspace which is left by removing the vector \a-) from V^r, i.e, 

VN/aj = span{|ai),..., \oij+i), • • •, |av)} (16) 

and let \ci^^^) ; n = 1,..., j — 1, j' + 1,..., iV be the corresponding reciprocal family which 
allows to express the orthogonal projector operator onto VN/aj as 

N N 

= £ \a^^^){an\. (17) 

n=l n=l 

n^j ruptj 

Assuming that the biorthogonal vectors ; n = 1,..., V yielding a representation of Pvn 
as given by 

N N 

I = E l«n)(«n| (18) 

n=l n=l 

are known, our goal is to modify such vectors so as to obtain the corresponding set \di ^^); n = 

1, • • •, j - 1, j + 1, ■ ■ ■, yielding as in (|^. 

We start by writing 


Pvn - A’Viv/c,. 


V, 


N/o 


(19) 


where is the orthogonal projector onto the orthogonal complement of V^/a^ 


m 


Vn- Thus, contains only one linear independent vector, arising by subtracting from |a^.) 

its component in i.e.. 


Pvi = 


( 20 ) 


where 


\P{) = l«,-) 


N/cj I'^j/ 


( 21 ) 


i''’7 = iiUk- 

(Note: we use the notation to differentiate this fresh vector from the previous |'0j) intro¬ 
duced in Section II.A) 

Using now (O). (|l^) and (^0|) we express (^^ as 
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( 22 ) 


N 


N 




n=l 


n = l 


Taking the inner product of both sides of (P^) with {^pj\, and using the fact that {^pj\an) = 0 
for n ^ j, we obtain: 


{di“j){“fi = {’iiy 


(23) 


Moreover, since (-0^ |ckj) = {oij |cxj) — {oij \ -Pvat/q \^j) ~ i'^j IV’/)> if follows from (|^) that 11 {^pj 111 = 
||(a;^|||“h Hence, vector {'ijj-!i\ turns out to be 




(a 


N\ 


(24) 


Taking now the inner product of both sides of (^) with every (a^l ; n = 1,... — + ... ,N 

we obtain the equation we wanted to hnd: 




= 1, ■ ■ ■, j - 1, j + 1, • • • 


(25) 


The following theorem demonstrates that the modihcation of vectors \a^) as prescribed in 
provides us with biorthogonal vectors ; n = 1,..., j —1, j + 1,..., iV rendering orthogonal 

projections. 

Theorem 1: Given a set of vectors la ^); ^ = 1, - ■ ■ ,N biorthogonal to vectors \an) ; n = 
1,..., iV and yielding a representation of /Vat cis given in (^), a new set of biorthogonal vectors 
1*5^) ;ri = — l,j + l,...,iV yielding a representation for , as given in ([T7|), can 

be obtained from the following equations 


= l«n ) 


\a 


V\||2 




(26) 


Proof: Let us hrst use (^^ to write 


N 


N 


pv„,„ = T. = i: \»n){»:\ - y. I“»)- ”'»^ 


n=l 

■n-itj 


71=1 


n=l 


\a 


W \||2 


(27) 


To prove that (|27|) is the orthogonal projector onto V]y/aj we show that a) Pv^/^. \g) = \g) for 
all l^f) G Visi/aj and b) Is'"*') = 0 for all in the orthogonal complement of Vat/q,^. in H. 

Indeed, every l^f) G Vat/q,. can be expressed as a linear combination l^f) = nn|<an), for some 
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coefficients ; n = 1,..., j — 1, j + 1,..., iV and since, by hypothesis, {a^\ai) = 5n,i from (|^) 


we have: 

Is') 

which proves a^ 

. 

To prove b) we 

write \a 

’n.^J 

(27) as 

Pvn/cj = Pvn 


= Pvn 


N N 


N 


-Pvv.js) = IZIZ|an)ai{a"l“i) = HonloJ = |g), 


(28) 


n = l 1 = 1 


n=l 


Pvjaf){af\ , 




\a 


Af\||2 


+ 


\a 


Ar\||2 




\a 


A^\l|2 


(29) 


Now, since {an\gj = 5nj{aj\g^) ] n = 1,..., A^, it follows that Py^gj = \aj){a-\g^) and, 
since \a^) G Vat, it follows that {af\g-^) = (d^liVjvIfi'"*') = Hence, 


= Pvn\9^) 




= = 0 - 


Af\||2 


(30) 


The biorthogonality property of vectors ; n = 1,..., j — 1, j + 1,..., iV is an immediate 

consequence of the biorthogonality property of vectors as readily follows by taking the 

inner product of both sides of (^) with each vector (q;„| ; n = 1,..., j — 1, j + 1,..., 

Since PvN/a ~ Pn){^n\ has been proved to be a projector, it is self-adjoint. Hence (|T7|) 

“i n^j 

holds □ 

Corollary 1: Let I/at) be the orthogonal projection of and arbitrary |/) G Ti onto Vat, i.e 


N 


(31) 


i/A') = ^’v;,i/) = i:c;;'ia„) 

n=l 

with = (oi^lf); n = 1,..., N assumed to be known. Hence, the coefficients of the 
orthogonal projection of |/) onto Vat/q^ are obtained from the known coefficients as follows: 


{a^\af)c^ 


„N/j 


3 I 3 


\a 


A^\l|2 


(32) 


The proof trivially stems from (|27|), since Ay,/„, I/) = implies = {a^/^f) □ 

'“j n^i 

Corollary 2: For |/) G 7i, let I/at) be as above and |/v/j) = A^iv/a l/)- Then, the following 
relation between |||/ai)|| and |||/Ar/j)|| holds: 









^N\2 


\\\fN/,)\\^=\\\fNW- 


\a 


N\\\2- 


(33) 


Proof: Using (PD|) and the fact that projectors are self-adjoint and idempotent, it follows that 


\\\fN/M? = U\Pv./.,\f) = U\Pv.\f) 

□ 




\a 


Af \||2 


= JN 


\a 


A'\l|2 


(34) 


So far we have discussed how to modify the coefficients of a linear expansion when one of its 
components is removed. Nevertheless, we have given no specihcation on how to choose such an 
element. We are now in a position to address this point, since the last Corollary suggests how 
the selection could be made optimal. The following proposition is in order. 

Proposition 1: Let 


N 


\!n) = PvJD = 


(35) 


n=l 


be given by the coefficients ; n = 1..., iV, and let 


N 




(36) 


n = l 


be obtained by eliminating the coefficient Cj from (|35D and modifying the remaining coefficients 
as prescribed in (|3^ . The coefficient to be removed as minimizing the norm of the residual 
error |A) = |/jv) — \fN/j) is the one yielding a minimum value of 

|„V|2 

(37) 


a 


n\\\2- 


Proof Since = Pyv/„ .^yv = 


Pvj^ = Pyv/„. we have: 

W) - |/w/,)|p = (/|PvJ/) - (/iPw.l/) = |||/iv)|p - |||/v/,)ir. 


(38) 


Hence, making use of (|3^) , we further have 


|„V|2 

lll/v)-|/iv/,)|P = P^, (39) 

from which we gather that IH/w) — \fN/j)\\‘^ is minimum if is minimum □ 

Proposition 1 is relevant to backward approximation of a signal, a common procedure in com¬ 
pression and noise reduction techniques. The goal being to shrink coefficients so as to have a 
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more economical representation and/or reduce spurious information (noise). Successive appli¬ 


cations of criterion (|37D leads to an algorithm for recursive coarser approximations. Indeed, 
let us assume that at the hrst iteration we eliminate the jth-term yielding a minimum of 
We then construct the new reciprocal vectors as prescribed in (^) and the corresponding new 
coefficients as prescribed in (p^. We are thus in a position to repeat the process and obtain a 
coarser approximation of the previous one. If we denote by the approximation arising at 

the fc-step, a common stopping criterion would recommend to cease the iteration process when 
the following situation is reached: 

||||/»-|/W)||=>i, (40) 


where 6 is estimated according to the desired precision. If the aim is to denoise a signal the 
value of 6 may be set as the variance of the noise, when available. It is appropriate to remark. 


however, that in the context of some applications the selection criterion (^) may not be the 
adequate one. Instead, other criteria based of statistical properties may be required 
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In any case, regardless of the criterion for selecting the coefficient to be overlooked, if one 
wishes the remaining ones to yield the optimal approximation in a minimum distance sense, such 
coefficients should be modihed as indicated in (|5^. We illustrate next, by a simple example, 
the gain that results in following this prescription. 

Let us consider = 13 elements |a„); n = 1,..., 13 whose functional representation are given 
by the following shifted Mexican hat wavelet 

a2n{t) = {t\a2n) = A2ne-- {t + 2 nf) 

a2n+i{t) = {t\a2n+i) = A2n+ie~^^~‘^'"~^^^{l-{t-2n-lf) ; n = 0,..., 6 (41) 


where each is a constant which normalizes the corresponding function |a„) to unity in the 
interval [—4,4]. We construct the biorthogonal functions (assumed to be known) by ap¬ 

plying the forward biorthogonalization technique of Section II.A, but they could be constructed 
by any other available method. 

The signal f(t) is considered to be the pick plotted by the continuous line of Figure 1. Such 
signal is also expressible by a linear combination of the waveforms given in (|4T|). A high qual¬ 
ity htting results by using the corresponding 13 coefficients c^, each of which is calculated as 
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cjf = (a^^l/). The actual numbers being the following: 

2.8273 2.4954 2.4954 1.9988 1.9988 1.4989 1.4989 
0.8630 0.8630 0.2957 0.2957 0.0648 0.0648 

We now disregard the last two coefficients, the ones of smallest magnitude, and use the re¬ 
maining ones without modihcation. Although the neglected coefficients are quite small in 
comparison with some of the others, the approximation that results, represented by the dotted 
line in Figure 1, does not £t correctly the distribution tails. Nevertheless, if we disregard the 
same coefficients but modify the others by applying ( |52D twice, the resulting approximation 
happens to coincide with the continuous line of Figure 1. To magnify the effect we wish to 
show, let us now disregard two more coefficients, those of value 0.2957. The approximation 
that results from a simple truncation is shown by the darker dotted line of Figure 2. The 
slender dotted line plots our approximation. This very simple example clearly illustrates the 
signihcance of the proposed modihcation of coefficients. 

IV. CONCLUSIONS 

A recursive approach for adapting biorthogonal functions so as to obtain orthogonal projec¬ 
tions onto a reduced subspace has been proposed. The required modihcations are simple and 
easy to implement. The modihed functions are used to adapt coefficients of a lower order linear 
model, in order to obtain an optimal approximation in a minimum distance sense. 

A criterion for disregarding coefficients has being discussed. Such criterion leads to an itera¬ 
tive procedure for successive backward approximations which yields, at each iteration, minimal 
residual norm. It should be stressed that, regardless of the criterion used for neglecting coeffi¬ 
cients, the proposed approach may be applied to guarantee optimality (in a minimum distance 
sense) of the remaining approximation. We believe, thereby, that this technique is potentially 
applicable to a broad range of problems including data compression, noise reduction and sparse 
representation. 
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FIGURES 
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Figure 1: The continuous line represents a signal f{t), which is also expressible as a linear 
combination of the 13 waveforms given in (i3)- The dotted line is the approximation arising 
by disregarding tow coefficients in such linear expansion. Our approach coincides with the 

continuous line. 
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Figure 2: Here the darker dotted line is obtained by disregarding four coefficients. Our 
approach is represented by the slender dotted line. 
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Figure Captions 


Figure 1: The continuous line represents a signal f{t), which is also expressible as a linear 
combination of the 13 waveforms given in (^T]). The dotted line is the approximation arising 
by disregarding tow coefficients in such linear expansion. Our approach coincides with the 
continuous line. 

Figure 2: Here the darker dotted line is obtained by disregarding four coefficients. Our 
approach is represented by the slander dotted line. 
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